Greg Detre
Tuesday, October 01, 2002
mentioned
to me by Deb Roy 021001,0945
Lera
BORODITSKY � Director, Cognation
������ brain and cognitive sciences
������ metaphorical influence of
lang on thought
������ math models
������ space/time
������ look at her recent stuff
overview
The citizens of cognation investigate the
relationships between mind, world, and language. How do nwe construct knowledge
from our experiences with the world? How do we use our knowledge to interpret
new experiences? In what ways have languages and cultures allowed us to go
beyond our innate capabilities and physical experiences to make us as smart as
we are?
how do we
mentally represent things we've never seen or touched?
what is the
relationship between language and thought?
how is
knowledge actually used in thought and action?
see:
Boroditsky & Ramscar, 'The roles of body and mind in abstract thought' in
Psychological Science, Vol 13 No 2 March 2002
They
conducted four studies, aiming to investigate the relationship between
cognitive representations of space and of time. They concluded that actual
movement through space, as well as just thinking about movement through space,
could affect how you think about time, i.e. whether you consider yourself as an
ego moving through time, or as a fixed point which times flows past/through.
They asked
people in different circumstances whether when faced with the normally
ambiguous statement, �Let�s move the Wednesday meeting forward by two days�,
they understood it to mean to Monday or to Friday.
She gives a
number of examples of experiments which show how different languages use on/in
vs tight/loose fit grammatical distinctions, relative/absolute spatial
references, shape vs material categorisations, and how grammatical gender can
influence your perceptions of nouns.
She does
cite an nice (but unsurprising) experiment with pre-linguistic infants, showing
how they are able to see the tight vs loose fit distinction before language
(either English or Korean) imposes its particular biases. Also though, if you
take a group of native English speakers and get them talking about time in a
vertical way (like Mandarin speakers) you soon start to get similar effects to
Mandarin speakers.
see �notes � reason, working ideas 9.doc�
Ray
Jackendoff considers the following sentences as using space and motion as a
metaphor for more abstract ideas. �Some deductions that apply to motion and
space also apply nicely to possession, circumstances and time�. We are not
merely co-opting words, but co-opting their �inferential machinery�.
The other fundamental set of metaphors in language
is force, agency and causation (c.f. the �impetus theory� underlying people�s
intuitive theory of physics). These concepts and relations of space and force
�appear to be the vocabulary and syntax of mentalese, the [combinatorial]
language of thought�. He speculates whether if �ancestral circuits for
reasoning about space and force were copied, the copy�s connections to the eyes
and muscles were severed and references to the physical world were bleached
out�, �the circuits could serve as a scaffolding whose slots are filled with
symbols for more abstract concerns like states, possessions, ideas and
desires�.
Lakoff and Johnsons�s
list of �metaphors we live by� include �argument is war�, �virtue is up�, �love
is a patient/force/madness/magic/war� and �ideas are
food/buildings/people/plants� etc.
Terry Regier and Laura Carlson, �Grounding spatial language in perception: An empirical and computational investigation� in Journal of Experimental Psychology: General, 2001. vol. 130(2), 273-298.
We ground the linguistic categorization of space in aspects of visual perception. Specifically, we ground the structure of projective spatial terms such as ``above'' in the process of attention, and in vector sum coding of overall direction. We formalize this in the attentional vector sum (AVS) model. This computational model accurately predicts linguistic acceptability judgments for spatial terms, under a variety of spatial configurations. In seven experiments, we test the predictions of the AVS model against those of three competing models. The results support the AVS model, and disconfirm its competitors. We conclude that the structure of linguistic spatial categories may be partially explained in terms of independently motivated perceptual processes.
�one reason to examine spatial language is that it offers the possibility of grounding some aspects of language in non-language�
�what non-linguistic perceptual processes may underlie the semantics of English projective spatial terms, such as 'above'?�
spatial template = (3D) x, y, mean acceptability
4 models of �above�
In English, time rushes forward. In Mandarin Chinese, it moves down. The
past lies above, and the future lies below.
she asked people to
answer simple time sequence questions while watching a video screen. When
objects on the screen move vertically, the Mandarin speakers are able to answer
faster than English speakers - implying that their brains processed time
questions differently, and hinting that there could be other differences
�And linguists also
came to realize that thoughts are much richer than language, undercutting the
very notion that people would need a word to think a thought.� - ???
She found a consistent pattern of German speakers using more masculine
terms to describe the key - such as ''hard, heavy, jagged'' - while Spanish
speakers favored more feminine descriptions, such as ''golden, intricate,
lovely.'' Boroditsky said she is now considering studying how the design of
bridges - a masculine word in Spanish, but a feminine word in German - differs
between the two cultures.
Now, though, the
research is turning to even more controversial ground, how speakers of
different languages remember events. Gleitman said she had just completed
research, accepted but not yet published by the journal Cognition, showing that
the different verb structures in English and Spanish do not cause speakers to
remember events differently.
But Boroditsky said
that she is beginning to uncover ''interesting differences'' in ongoing
research into how speakers of Turkish and other languages remember events.
"We as
a species have eyes because it prevents us from going around and licking
everything all the time," boroditsky says. "We can explore things
from a distance. Everything we know about the world comes in through two little
holes in our heads."
"She
likes bananas because she believes they're directly responsible for humans
seeing in color - evolving monkeys had to be able to spot fruit on trees in
order to avoid starving."
What if all this talk makes us think twice
about relying on our vision for everything, a freaked-out audience member asks?
"Licking always works," she says. "When all else fails, go back
to licking."
For one,
some noted that color perception is probably too biologically ingrained to show
influence from language.
Mayan
speakers, on the other hand, do not refer to objects in plural form, so shape
and unit are less ingrained into their speech. Accordingly, their language
revolves more around what objects are made of than English; a "candle" to English speakers is a "long, thin wax"
to Mayans. � John Lucy, Univ of Chicago
To see if the thought and speech patterns of the
two groups coincided, Lucy presented individuals with an object such as a comb
or box. He asked study subjects to decide which of two other objects was more
similar--one with the same shape but made of a different material or vice
versa. The groups' preferences split along linguistic lines. English speakers fancied shape; Mayans liked material. Lucy and a
coworker then found that Mayan children shared the English predilection for
shape until age seven or so, but turned toward material by age nine. This cognitive
difference appeared after the children had acquired language, suggesting that
their thought patterns diverged as they acclimated to their way of speaking.
To give two
different-language groups the same test, you have to translate it. "But
once you've translated the test, you no longer know if you've got the same
test," Boroditsky explains. So she has focused on bilinguals.
Even toasters have gender in our private
thoughts, according to one study. Bilingual individuals shown pictures of
objects and people rated the objects more similar (on a scale of one to four)
to girls, ballerinas and other inherently feminine words if the objects were
grammatically feminine in their native language. So too, they found objects
more similar to inherently masculine words if the object names were masculine.
Native English speakers showed similar patterns
after they learned the grammar system of a made-up language. In other words,
just a brief change in the way people talk can create a measurable effect,
Boroditsky says.
She next examined how English speakers compared
with Mandarin-English bilinguals in thinking about time. English speakers tend
to talk about time in terms of horizontal dimensions: for example, the meeting
was moved "forward" or "back." In Mandarin, however, next
month is "down" the calendar and last month is "up." As you
might expect would be the case if the two groups think about time differently,
the bilinguals figured out if one event preceded another faster after
concentrating on a vertical stimulus, and the English-only group benefited from
horizontal cues. Moreover, those in the Mandarin group who learned English
later in life tended to have a stronger vertical bias. Indicating that it
wasn't necessarily the Mandarin convention of writing vertically that caused
the effect, English speakers trained briefly to talk about time using vertical
metaphors showed more Mandarin-like results on the same tests. "That's a
really powerful effect of language on thought," Boroditsky concludes, and
one that shows how flexible our minds are.
Maybe
language itself gives us some of this flexibility, says Gentner, particularly
when it comes to grasping relationships. She and a colleague tested three- to
four-year-old kids with a hide-and-seek game that used two matching
three-tiered shelves. On each tier was an identical plastic pig, and one of the
pigs had a toy hidden inside. The child got to see where the winning pig was in
one shelf. Then, to find the toy, he or she had to choose the pig in the same
relative position in the other shelf--a challenging analogy for a child that
age, Gentner notes. The kids often lost track of the winning position and
searched randomly. When the test administrator started the game by naming the
three locations--top, middle and bottom--the kids chose the correct location
far more often, so the spatial terms helped them remember. For difficult
problems, then, the proper use of language can invite us to think in a more
productive way, Gentner says.
the Regier
model is one of the better examples of a mathematical model of spatial
semantics
let's assume he's approximately correct in some of the features that
he's extracting
we know
that we definitely reuse spatial terms to describe temporal
what Boroditsky showed is that this is not just some sort of
coinicidence or loose metaphor, but that they're tightly connected - and if you
change the way people think spatially, it affects the way they think temporally
develop some models of that connection
start with something like a Regier model, very concrete, and then play
with it, and see how you can hook it in to the temporal and put it to some use,
and maybe replicate some of the behaviour
Lera
show them pictures (non-linguistic) with
languages that do and don�t mark tenses � if you�re an English speaker you�re
much more likely to confuse people, but if you�re an Indonesian speaker, you�re
much more likely to make tense errors
she always does her experiments bi-directionally,
but there always seems to be a source + target
regier +
carlson
wouldn�t you want to learn from raw data???
you run into trouble � always need some kind of starting representation
the blank slate idea just doesn�t hold water
questions
you want to decide for your learning system, e.g. for a NN
representation � what encoding you use
structure/learning algorithm � learning rule
parameter estimation � values of weights
usually, when people talk about machine
learning, they�re just talking about the third � though sometimes they�re
talking about structure learning
internal representation � you�ve already made
some decisions as an engineer in terms of what you feed it
PDP group � wickelgreen??? past tense verb
inflections learnt by connectionist systems
the lera
space-time project is more about analogy-making than learning
how do you draw those analogies except by some sort of learning
system???
how can you put the geometric represnetation to use for temporal
represntation
paper on the semantics of �in� � words like �ahead� and �beside�
how early
do words like �above� show up developmentally???
can we teach them to chimps???
see Landau + Jackendorff (1993) BBS � the �what� and the
�where� of the spatial �??? or something like that � separating out shape and �
- see Google
thinking about language as a lens through which to examine the visual
system
what are the minimum set of visual primitive sthat the visual system
must be encoding so that we cn tlak about what we see
has a good list of spatial relation terms
they�ve
already implemented the Regier paper (the features)
trying to standardise the way they implement visual processing
create a world where you can extract features from simulated scenes, and
print up some models using that feature set
take the features that have been implemented and the framework in which
we talk to those features � use some sort of simple statistical machine
learning to learn a couple of spatial models (e.g. ahead (has some notion of
directionality), next, near???), collect some data with a landmark and
trajector, feed into the system, get back the regier features, see if you�ve
trained a reasonable model for whatever spatial terms you�ve picked � at that
point we�ll have another conversation
start thinking about how you would reuse that model
starting idea:
2D world, with some model of next, project onto 1D world, using that
projection as the starting point for how you ground the meaning of next
objects become intervals, relations between objects become intervals on
a timeline
Lera�s intuition � don�t project � keep it in 2D
if you�re not projecting, then what exactly are you doing when moving
from the spatial to temporal domain???
I guess you�re presenting things in parallel, then in sequence � perhaps
you�re having the two representations at once
semantics of the temporal relation �next�
behaviourally � show them a movie and ask them to describe it � give
them several moveies with a description and ask them which fits it
e.g. red flash is next to the blue flash
linguistic behavioural test that you can ask the machine to do,
describing/labelling sequ4ences, using its vocabulary (receptive or productive)
kind of like kids when you�re testing their language skills
how do you put it to use???
if you�ve only got one object in your video sequence, then �
look at
kids language test papers
look at
more of lera�s papers
semantics
of �in� paper
talk to
peter
is the
question about moving a meeting forward normally a 50-50 intuitive split???
what other
circumstances affect how people answer this question???
what sort
of representation(s) do they speculate might underly this phenomenon??? if we
were building a cognitive system which represented space + time in some
parallel way in the hope of demonstrating a similar result, how should we go
about it???
do we know
for sure that space is the more fundamental of the two relations???
is it not at least as plausible that time is the more
fundamental�
do they even/actually say that space is the more fundamental one??? do
they even talk in terms of one being more fundamental than the other???
how can we look at how one has affected the other???
space and
time have different dimensionalities (presumably) � might we expect that this
dimension-reduction might affect the representation of time somehow???
for instance, she mentions Mandarin and English speakers as utilising
horizontal vs (mainly) vertical timelines
given that space and time two have different dimensionalities (3 vs
presumably 1), then might there be other/easier representations that the
temporal sense could also/instead be co-opting??? like what??? see Strawson on
how fundamental the notion of space is to the notion of difference/distance�
I ought to read her paper on comparison to see if there�s some link to
be found between space and difference�
is it possible that we might be able to discern some vestigial
3-dimensionality in our temporal representation???
she�s in
the brain + cognitive science dept � roughly roughly how do we imagine that
this space/time shared/borrowed/over-lapping representation might work � would
it be a genetic thing???
do they envisage the representations as being shared vs borrowed vs
over-lapping???
surely we
should expect that there are different types of spatial representations???
indeed, we know that motor commands are represented to different levels of
detail in different parts of the motor cortex, so presumably there�s more than
one spatial representation too
is she saying that at some stage in our evolutionary (or even
developmental) past, the time-representation grew out of/co-opted our mental
space-representation structure???
or is she saying that in some way the two are still linked, still
co-located in neural space, and so thinking about space influences thinking
about time???
if it is the latter she�s saying, then shouldn�t we be able to test this
by getting people to think about time and seeing how it affects their thoughts
about space???
if we accept that space and time do share a representation:
what other co-opted representations might there be???
e.g. Lakoff and Johnsons�s list of �metaphors we
live by� include �argument is war�, �virtue is up�, �love is a
patient/force/madness/magic/war� and �ideas are food/buildings/people/plants�
etc.
how can we look at how the co-opting occurs???
does it depend specifically on the nature of the initial representation, and the task to which the new representation will be applied, or can we generalise???
how easy is it???
can we imagine a genetic mechanism for it??? doesn�t Pinker consider that it would be a pretty easy thing to do, given that he thinks that certain neural architectures are specifed in the genes somewhere broadly, only requiring that some string of genes get processed twice etc.???
what alternatives to the idea that space and time representations have some linkages are there???
is this a quirk of evolution, or might it in fact conceivably/ever be otherwise???
what would it take to discredit the notion???
is this the sort of thing that Minsky would endorse???
boroditsksy - linguistic relativity - can't they branch into less basic/perceptuomotor linguistic relativisms???
i know this is hard to demonstrate experimentally, but we're interested in concepts, not immediate sensory representations. the shape vs material preference, like the gender-based adjective preferences does go some way towards this though�
look at my pinker notes on lakoff(???) � see questions + summary above
with the experiment which teaches English speakers to use vertical time words like Mandarin, presumably this is thought to show that language�s effects on concepts are immediate and plastic�???
but couldn�t it also show simply/perhaps that both conceptual representations are present at all time, but only get filtered/affected when/after you start to talk about them�???
but what about memory
are the first three models straw men??? I haven�t seen the fourth yet. it all feels pretty obvious
would we ever want to specify the exact nature of the spatial representation in this much detail??? I suppose we probably would, unless we were being really hands-off and self-organising
no, I know what I�m trying to ask � I�m wondering why/whether we�d ever want to build in a feature/definition/sense/model of something like �above� � surely, we�d want that to be a concept that is implicitly represented in whatever (world-centred/allocentric) schema you have of space, that your linguistic concepts can attach to/bring out/
would Stringer�s place coding stuff be relevant??? isn�t place coding more to do with absolute references???